将知识蒸馏应用于个性化的跨筒仓联合学习，可以很好地减轻用户异质性的问题。然而，这种方法需要一个代理数据集，这很难在现实世界中获得。此外，基于参数平均的全球模型将导致用户隐私的泄漏。我们介绍了一个分布式的三位玩家GaN来实现客户之间的DataFree共蒸馏。该技术减轻了用户异质性问题，更好地保护用户隐私。我们证实，GaN产生的方法可以使联合蒸馏更有效和稳健，并且在获得全球知识的基础上，共蒸馏可以为各个客户达到良好的性能。我们对基准数据集的广泛实验证明了与最先进的方法的卓越的泛化性能。

SoftMax函数广泛用于人工神经网络，用于多级分类问题，其中SoftMax变换强制执行输出为正和总和，并且相应的损耗功能允许使用最大似然原理来优化模型。然而，Softmax留下了大幅的损失函数，以便在高维分类方面进行优化操作，这导致在一定程度上的低性能。在本文中，我们提供了一种对简单简洁的软制态变体，即稀疏-Softmax的实证研究，以减轻在高维分类问题方面的传统软邮件中发生的问题。我们在几个跨学科任务中评估了我们的方法，实验结果表明，Soparse-SoftMax更简单，更快，并产生比基线模型更好的结果。

Machine learning models can reach high performance on benchmark natural language processing (NLP) datasets but fail in more challenging settings. We study this issue when a pre-trained model learns dataset artifacts in natural language inference (NLI), the topic of studying the logical relationship between a pair of text sequences. We provide a variety of techniques for analyzing and locating dataset artifacts inside the crowdsourced Stanford Natural Language Inference (SNLI) corpus. We study the stylistic pattern of dataset artifacts in the SNLI. To mitigate dataset artifacts, we employ a unique multi-scale data augmentation technique with two distinct frameworks: a behavioral testing checklist at the sentence level and lexical synonym criteria at the word level. Specifically, our combination method enhances our model's resistance to perturbation testing, enabling it to continuously outperform the pre-trained baseline.

这项工作研究了一个由机器人辅助的人群疏散问题，我们控制了一小群机器人，以指导大量的人群到安全地点。挑战在于如何建模人类机器人的相互作用和设计机器人控制以间接控制人口，从而超过了机器人。为了应对挑战，我们将人群视为连续体，并将疏散目标提出，以推动人群密度到目标位置。我们提出了一个新型的均值模型，该模型由一个微观方程组成，该系列明确模拟了人类运动如何由机器人局部指导和相关的宏观方程，该方程描述了如何由所有机器人产生的人群密度控制的人群密度。 。然后，我们为机器人设计密度反馈控制器，以动态调整其状态，以使生成的导航速度字段将人群密度驱动到目标密度。证明了拟议控制器的稳定性保证。包括基于代理的模拟来评估所提出的疏散算法。

在本文中，我们研究了一群代理的旨在通过流数据进行协作地学习共同的静态潜在函数的问题。我们提出了一种轻量级分布式高斯进程回归（GPR）算法，该算法是在通信，计算和内存中的代理有限能力的认识。每个代理使用本地流数据独立地运行基于代理的GPR，以预测感兴趣的测试点;然后，该代理协作执行分布式GPR，以获得通过常见的稀疏测试点集的全局预测;最后，每个代理的融合来自分布式GPR的结果与基于代理的GPR来改进其预测。通过量化预测方差和错误中的瞬态和稳态性能，我们表明，有限的代理商通信在帕累托的意义上提高了学习表演。Monte Carlo仿真进行了评估发达的算法。

Masked image modeling (MIM) performs strongly in pre-training large vision Transformers (ViTs). However, small models that are critical for real-world applications cannot or only marginally benefit from this pre-training approach. In this paper, we explore distillation techniques to transfer the success of large MIM-based pre-trained models to smaller ones. We systematically study different options in the distillation framework, including distilling targets, losses, input, network regularization, sequential distillation, etc, revealing that: 1) Distilling token relations is more effective than CLS token- and feature-based distillation; 2) An intermediate layer of the teacher network as target perform better than that using the last layer when the depth of the student mismatches that of the teacher; 3) Weak regularization is preferred; etc. With these findings, we achieve significant fine-tuning accuracy improvements over the scratch MIM pre-training on ImageNet-1K classification, using all the ViT-Tiny, ViT-Small, and ViT-base models, with +4.2%/+2.4%/+1.4% gains, respectively. Our TinyMIM model of base size achieves 52.2 mIoU in AE20K semantic segmentation, which is +4.1 higher than the MAE baseline. Our TinyMIM model of tiny size achieves 79.6% top-1 accuracy on ImageNet-1K image classification, which sets a new record for small vision models of the same size and computation budget. This strong performance suggests an alternative way for developing small vision Transformer models, that is, by exploring better training methods rather than introducing inductive biases into architectures as in most previous works. Code is available at https://github.com/OliverRensu/TinyMIM.

In this paper, we propose a robust 3D detector, named Cross Modal Transformer (CMT), for end-to-end 3D multi-modal detection. Without explicit view transformation, CMT takes the image and point clouds tokens as inputs and directly outputs accurate 3D bounding boxes. The spatial alignment of multi-modal tokens is performed implicitly, by encoding the 3D points into multi-modal features. The core design of CMT is quite simple while its performance is impressive. CMT obtains 73.0% NDS on nuScenes benchmark. Moreover, CMT has a strong robustness even if the LiDAR is missing. Code will be released at https://github.com/junjie18/CMT.

Dataset distillation has emerged as a prominent technique to improve data efficiency when training machine learning models. It encapsulates the knowledge from a large dataset into a smaller synthetic dataset. A model trained on this smaller distilled dataset can attain comparable performance to a model trained on the original training dataset. However, the existing dataset distillation techniques mainly aim at achieving the best trade-off between resource usage efficiency and model utility. The security risks stemming from them have not been explored. This study performs the first backdoor attack against the models trained on the data distilled by dataset distillation models in the image domain. Concretely, we inject triggers into the synthetic data during the distillation procedure rather than during the model training stage, where all previous attacks are performed. We propose two types of backdoor attacks, namely NAIVEATTACK and DOORPING. NAIVEATTACK simply adds triggers to the raw data at the initial distillation phase, while DOORPING iteratively updates the triggers during the entire distillation procedure. We conduct extensive evaluations on multiple datasets, architectures, and dataset distillation techniques. Empirical evaluation shows that NAIVEATTACK achieves decent attack success rate (ASR) scores in some cases, while DOORPING reaches higher ASR scores (close to 1.0) in all cases. Furthermore, we conduct a comprehensive ablation study to analyze the factors that may affect the attack performance. Finally, we evaluate multiple defense mechanisms against our backdoor attacks and show that our attacks can practically circumvent these defense mechanisms.

Blind image quality assessment (BIQA) remains challenging due to the diversity of distortion and image content variation, which complicate the distortion patterns crossing different scales and aggravate the difficulty of the regression problem for BIQA. However, existing BIQA methods often fail to consider multi-scale distortion patterns and image content, and little research has been done on learning strategies to make the regression model produce better performance. In this paper, we propose a simple yet effective Progressive Multi-Task Image Quality Assessment (PMT-IQA) model, which contains a multi-scale feature extraction module (MS) and a progressive multi-task learning module (PMT), to help the model learn complex distortion patterns and better optimize the regression issue to align with the law of human learning process from easy to hard. To verify the effectiveness of the proposed PMT-IQA model, we conduct experiments on four widely used public datasets, and the experimental results indicate that the performance of PMT-IQA is superior to the comparison approaches, and both MS and PMT modules improve the model's performance.

Automatic music generation with artificial intelligence typically requires a large amount of data which is hard to obtain for many less common genres and musical instruments. To tackle this issue, we present ongoing work and preliminary findings on the possibility for deep models to transfer knowledge from language to music, by finetuning large language models pre-trained on a massive text corpus on only hundreds of MIDI files of drum performances. We show that by doing so, one of the largest, state-of-the-art models (GPT3) is capable of generating reasonable drum grooves, while models that are not pre-trained (Transformer) shows no such ability beyond naive repetition. Evaluating generated music is a challenging task, more so is evaluating drum grooves with little precedence in literature. Hence, we propose a tailored structural evaluation method and analyze drum grooves produced by GPT3 compared to those played by human professionals, exposing the strengths and weaknesses of such generation by language-to-music transfer. Our findings suggest that language-to-music transfer learning with large language models is viable and promising.